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Background / Context: 

Relatively few rigorous studies of the effectiveness of contemporary public pre- 
kindergarten programs have been conducted despite the growing number of programs and large 
monetary investments that accompany them. The study on which this presentation is based was 
launched in partnership with the Tennessee State Department of Education’s Division of School 
Readiness and Early Learning to provide an assessment of the effects of the statewide Tennessee 
Voluntary Pre-Kindergarten (TN-VPK) program on the readiness for kindergarten of the 
economically disadvantaged population it serves. TN-VPK has become a controversial program 
in Tennessee, with some legislators expressing doubts about its value in the context of severe 
budget shortfalls and still others referring to it even more skeptically as expensive babysitting. 
This study interleaves a randomized control trial (RCT) design and an age-cutoff regression 
discontinuity (RD) design to evaluate the effectiveness of the TN-VPK program. Though the 
project is still underway, this presentation will summarize results from the first two years. 

Purpose / Objective / Research Question / Focus of Study: 

This study includes a randomized control experiment, but that component could only be 
implemented in a limited number of schools with more applicants than seats in the pre-k 
program. Informative as that component is about the effects of TN-VPK, the participating 
schools do not provide a representative sample of the TN-VPK classrooms in Tennessee. To 
provide a more representative statewide picture, a stratified random sample of schools with TN- 
VPK classrooms was drawn and enrolled in an age cutoff regression-discontinuity designs. This 
design was made possible because TN-VPK has explicit age requirements that are implemented 
as a strict age cutoffs for TN-VPK eligibility in all schools. 

Using the two designs concurrently, this study seeks to determine the statewide effect of 
TN-VPK on both the kindergarten readiness of the participating children and their long-term 
cognitive and behavioral skills, including performance on the state-administered achievement 
tests in the third grade. A secondary purpose is to investigate the relationships between those 
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outcomes and the characteristics of the TN-VPK classrooms, e.g., the curriculum used, teacher 
credentials, and classroom quality, and structural support and monitoring at the district and 
school levels, detecting impact heterogeneity by these characteristics. 

Setting: 

TN-VPK supports over 900 pre-k classrooms that serve more than 18,000 at-risk children 
in 133 of the 136 Tennessee school systems. It is, therefore, truly a statewide program serving all 
areas of the state, both rural and urban. 

Population / Participants / Subjects: 

To be eligible for TN-VPK, children must be age four on or before September 30 of the 
respective school year. By statutory requirement, the program gives top priority to children who 
qualify for the Free or Reduced Price Lunch Program, and 86% of the children enrolled 
statewide meet that criterion. Statewide, 49% of the children enrolled in TN-VPK are 
racial/ethnic minorities (non- White), 4% are English language learners, and 11% have special 
education designations. 

Intervention / Program / Practice: 

TN-VPK operates through competitive grants to local school systems who apply for 
approval and funding. Those grants support only a portion of the necessary classroom funding, 
the balance must come from other sources. This arrangement permits and encourages 
collaboration between school systems and other organizations. In this “collaboration model,” 
school districts may operate their pre-k programs through collaborative agreements with local 
nonprofit and for-profit child care providers and Head Start programs so long as those agencies 
have attained the highest rating from the licensing system administered by the Tennessee 
Department of Human Services and meet the TN-VPK standards. Those standards, which are set 
by the State Board of Education, require the following: 

• A state licensed teacher with an early childhood education endorsement in each classroom; 

• A teacher assistant who holds or is actively working toward at least a CDA or associate 
degree in early childhood; 

• Professional development support for teachers; 

• An adult- student ratio no smaller than 1:10; 

• A small class size maximum of 20; 

• An approved age- appropriate curriculum aligned with the Tennessee Early Childhood 
Education Developmental Standards; 

• A family engagement component and a pre-k to kindergarten transition plan for each child; 

• Vision, hearing, and health screening and referral services; 

• A minimum of 5.5 hours per day, exclusive of nap time, for a minimum of 180 days per year 
within a calendar that includes 200 working days of 7.5 hours for teaching staff. 

Research Design: 

RCT. In the summer before each of two school years, administrators in schools that 
expected more applicants to their TN-VPK program than available slots were asked to participate 
in the RCT. All eligible applicants were placed on a list that was randomized by the research 
team. Children were offered seats in the VPK program in the order that they appeared on the 
randomized list. This procedure gave each child an equal chance to be ranked high enough on the 
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list to be admitted but, also, by the same equal chance, left some children too low on the list for a 
seat to be available for them. The total sample across both cohorts included more than 2,000 
children. A subsample of these randomized children was consented and assessed at the beginning 
and end of their VPK year, at the end of kindergarten, and will be assessed at the end of first, 
second, and third grade, as well. Because there are more applicants than seats in only some areas 
of the state, the RCT has the disadvantage that the participating schools are not necessarily 
representative of the TN-VPK program statewide. The RDD component compensates for that 
disadvantage. 

RDD. A stratified random sample of 128 TN-VPK classrooms was drawn from the 2009- 
2010 state population of 942 classrooms in a manner that ensured adequate representation of the 
geographical regions of Tennessee, urban and rural schools, and public school based and 
partnership programs implemented collaboratively with community agencies. No more than one 
classroom per school/site was selected and each classroom was designated a block in a blocked 
RD design; that is, each classroom provides both treatment and control cases from two 
successive cohorts of students. One of four designated Tennessee regions enters the design each 
year of the project, with the first region beginning in school year 2009-2010. Each cohort of 
students is assessed in the fall early in the school year with the treatment group assessed at the 
beginning of kindergarten after participating in the selected TN-VPK classroom the prior year 
and the control group of children assessed that same fall soon after they are newly enrolled in 
that classroom. During the school year when the treatment cohort is in the selected classroom, 
classroom observations measures are taken along with teacher questionnaires and other 
descriptive information about the respective classroom. A primary disadvantage of this RDD 
designs is that it precludes longitudinal follow-up, as it lacks a comparison group that never 
attends VPK. The RCT portion of the study allows for such long-term analyses, however. 

Data Collection and Analysis: 

Children in both parts of the study are individually assessed using a set of Woodcock 
Johnson III achievement tests of pre-reading, language, and mathematic skills (Letter-Word 
Identification, Spelling, Understanding Directions, Applied Problems, Quantitative Concepts, 
Passage Comprehension, and Oral Comprehension). In the RCT portion, teacher ratings of 
children’s cognitive, social, and behavioral skills were also collected. In the RDD portion, 
classroom observations are conducted using the ECERS and a locally developed instrument 
(‘narrative record’) on which classroom activities and the time spent on each are recorded. 
Analysis is done multilevel with children nested in classrooms. At the time of this writing, RCT 
data are available for the first cohort of children through kindergarten, representing 287 children 
who received VPK and 91 children who did not, and for the second cohort of children through 
the end of the VPK year, representing 600 children who received VPK and 288 children who did 
not. Data collection is currently underway for kindergarten teacher ratings for the second cohort, 
and kindergarten assessments for that cohort will begin in the new year. RDD data are available 
from the first of the four Tennessee regions representing 32 classrooms and more than 1200 
children in the two cohorts enrolled in those classrooms. Data collection is currently underway 
for classrooms in the second and third regions. 

Significance / Novelty of study: 

This study will ultimately produce a large, representative statewide sample for which pre- 
k effects can be analyzed from both an experimental and a rigorous quasi-experimental 
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perspective. There is some overlap between the schools participating in the randomized design 
and those participating in the regression-discontinuity design. These are not perfectly aligned so 
as to involve the same treatment children and school years but, in many cases, are close enough 
to permit potentially informative comparisons between the treatment estimates derived from the 
randomized design and those from the regression-discontinuity design. These two perspectives 
combine to yield one of the largest, most rigorous explorations of the effects of a public pre- 
kindergarten program to date. 

Findings / Results: 

Though data collection is continuing in both components of this study, some outcome 
data are already available for analysis. The results of the RCT so far indicate that children who 
participated in TN-VPK significantly outperformed the children who did not attend TN-VPK at 
the end of the VPK year on all of the direct assessment scales examined, with effect sizes 
ranging from .28 to .42. Additionally, there were statistically significant differences favoring the 
children who had attended TN-VPK on the kindergarten teachers’ ratings of kindergarten 
readiness and work-related skills. There were no differences on the other teacher rated measures. 
The results of the RCT were supported by data analysis of the RDD component. The results of 
the RDD study so far indicate that children who participated in TN-VPK significantly 
outperformed children who did not attend the program on all direct assessments examined. 
Further analysis of newly acquired outcome data from the second year of both the RCT and RDD 
is currently underway. Results from analyses examining both the effectiveness of the overall TN- 
VPK program as well as the influence of certain program characteristics will be available for this 
presentation at the time of the SREE conference. 



2012 SREE Conference Abstract Template 



4 




